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1. Scope* 


1.1 This practice covers the establishment and maintenance 
of the essentials of a quality management system in laborato- 
ries engaged in the analysis of petroleum products, liquid fuels, 
and lubricants. It is designed to be used in conjunction with 
Practice D6299. 

Note 1—This practice is based on the quality management concepts 
and principles advocated in ANSI/ISO/ASQ Q9000 standards, ISO/IEC 
17025, ASQ Manual,” and ASTM standards such as D3244, D4182, 
D4621, D6299, D6300, D7372, E29, E177, E456, E548, E882, E994, 
E1301, E1323, STP 15D,* and STP 1209.* 

1.2 This standard does not purport to address all of the 
safety concerns, if any, associated with its use. It is the 
responsibility of the user of this standard to establish appro- 
priate safety and health practices and determine the applica- 
bility of regulatory requirements prior to use. 

1.3 This international standard was developed in accor- 
dance with internationally recognized principles on standard- 
ization established in the Decision on Principles for the 
Development of International Standards, Guides and Recom- 
mendations issued by the World Trade Organization Technical 
Barriers to Trade (TBT) Committee. 


2. Referenced Documents 


2.1 ASTM Standards:> 
D86 Test Method for Distillation of Petroleum Products and 
Liquid Fuels at Atmospheric Pressure 


' This practice is under the jurisdiction of ASTM Committee D02 on Petroleum 
Products, Liquid Fuels, and Lubricants and is the direct responsibility of Subcom- 
mittee D02.94 on Coordinating Subcommittee on Quality Assurance and Statistics. 

Current edition approved May 1, 2017. Published June 2017. Originally 
approved in 2002. Last previous edition approved in 2013 as D6792 — 13. DOI: 
10.1520/D6792-17. 

? “Quality Assurance for The Chemical and Process Industries: A Manual of 
Good Practices,” 1987, available from American Society for Quality (ASQ), 600 N. 
Plankinton Ave., Milwaukee, WI 53203. www.asq.org. 

3 ASTM STP 15D, ASTM Manual on Presentation of Data and Control Chart 
Analysis, ASTM International, W. Conshohocken, PA. 

4 ASTM STP 1209, ASTM Manual on Total Quality Management, ASTM 
International, W. Conshohocken, PA. 

5 For referenced ASTM standards, visit the ASTM website, www.astm.org, or 
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM 
Standards volume information, refer to the standard’s Document Summary page on 
the ASTM website. 


*A Summary of Changes section appears at the end of this standard 


Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States 


www.astm.org. 


D3244 Practice for Utilization of Test Data to Determine 
Conformance with Specifications 

D4057 Practice for Manual Sampling of Petroleum and 
Petroleum Products 

D4182 Practice for Evaluation of Laboratories Using ASTM 
Procedures in the Sampling and Analysis of Coal and 
Coke (Withdrawn 2010)° 

D4621 Guide for Quality Management in an Organization 
That Samples or Tests Coal and Coke (Withdrawn 2010)° 

D5191 Test Method for Vapor Pressure of Petroleum Prod- 
ucts (Mini Method) 

D5842 Practice for Sampling and Handling of Fuels for 
Volatility Measurement 

D5854 Practice for Mixing and Handling of Liquid Samples 
of Petroleum and Petroleum Products 

D6299 Practice for Applying Statistical Quality Assurance 
and Control Charting Techniques to Evaluate Analytical 
Measurement System Performance 

D6300 Practice for Determination of Precision and Bias 
Data for Use in Test Methods for Petroleum Products and 
Lubricants 

D6617 Practice for Laboratory Bias Detection Using Single 
Test Result from Standard Material 

D6708 Practice for Statistical Assessment and Improvement 
of Expected Agreement Between Two Test Methods that 
Purport to Measure the Same Property of a Material 

D7372 Guide for Analysis and Interpretation of Proficiency 
Test Program Results 

E29 Practice for Using Significant Digits in Test Data to 
Determine Conformance with Specifications 

E177 Practice for Use of the Terms Precision and Bias in 
ASTM Test Methods 

E456 Terminology Relating to Quality and Statistics 

E548 Guide for General Criteria Used for Evaluating Labo- 
ratory Competence (Withdrawn 2002)° 

E882 Guide for Accountability and Quality Control in the 
Chemical Analysis Laboratory 

E994 Guide for Calibration and Testing Laboratory Accredi- 
tation Systems General Requirements for Operation and 


©The last approved version of this historical standard is referenced on 
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Recognition (Withdrawn 2003)° 

E1301 Guide for Proficiency Testing by Interlaboratory 
Comparisons (Withdrawn 2012)° 

E1323 Guide for Evaluating Laboratory Measurement Prac- 
tices and the Statistical Analysis of the Resulting Data 

2.2 ISO Standards:' 

ISO Guide 30 Terms and Definitions Used in Connection 
with Reference Materials 

ISO/IEC 17025 General Requirements for the Competence 
of Testing and Calibration Laboratories 

ISO 4259 Petroleum Products—Determination and Applica- 
tion of Precision Data in Relation to Methods of Test 

ANSI/ISO/ASQ Q9000 Quality Management System Stan- 
dards 

2.3 Other Standards: 

40 CFR 80 Regulation of Fuels and Fuel Additives® 


3. Terminology 


3.1 Definitions: 

3.1.1 accepted reference value, ARV, n— a value that serves 
as an agreed upon reference for comparison, and which is 
derived as: (/) a theoretical or established value, based on 
scientific principles, (2) an assigned value, based on experi- 
mental work of some national or international organization 
such as the U.S. National Institute of Standards and Technol- 
ogy (NIST), or (3) a consensus value, based on collaborative 
experimental work under the auspices of a scientific or 
engineering group. E456 


3.1.2 accuracy, n—the closeness of agreement between a 
test result and an accepted reference value. E456 


3.1.3 audit, n—a systematic examination of a laboratory’s 
quality management system documentation and related activi- 
ties by an internal or external team to determine conformance 
to the applicable quality management system standard, such as 
described in this practice. 


3.1.4 bias, n—the difference between the population mean 
of the test results and an accepted reference value. E456 


3.1.5 calibration standard, n—a material with a certified 
value for a relevant property, issued by or traceable to a 
national organization such as NIST, and whose properties are 
known with sufficient accuracy to permit its use to evaluate the 
same property of another sample. 


3.1.6 certified reference material, CRM, n— a reference 
material one or more of whose property values are certified by 
a technically valid procedure, accompanied by a traceable 
certificate or other documentation which is issued by a certi- 
fying body. ISO Guide 30 


3.1.7 measurand, n—the measurable quantity subject to 
measurement. 


3.1.8 outlier, n—a result far enough in magnitude from other 
results so as to be considered not a part of the set. D6300 


7 Available from American National Standards Institute (ANSI), 25 W. 43rd St., 
4th Floor, New York, NY 10036, http://www.ansi.org. 

8 Available from U.S. Government Printing Office, Superintendent of 
Documents, 732 N. Capitol St., NW, Washington, DC 20401-0001, http:// 
WWW.aCCeSs.gpo. gov. 


3.1.9 precision, n—the closeness of agreement between test 
results obtained under prescribed conditions. E456 


3.1.10 proficiency testing, n—determination of a laborato- 
ry’s testing capability by evaluating its test results in interlabo- 
ratory exchange testing or crosscheck programs. 

3.1.10.1 Discussion—One example is the ASTM D02 com- 
mittee’s proficiency testing programs in a wide variety of 
petroleum products and lubricants, many of which may involve 
more than a hundred laboratories. 


3.1.11 quality assurance (QA), n—a system of activities, the 
purpose of which is to provide to the producer and user of a 
product, measurement, or service the assurance that it meets 
the defined standards of quality with a stated level of confi- 
dence. 

3.1.11.1 Discussion—Quality assurance includes quality 
planning and quality control. 


3.1.12 quality control (QC), n—a planned system of activi- 
ties whose purpose is to provide a level of quality that meets 
the needs of users; also the uses of such a system. 


3.1.13 quality control sample (QC sample), n— for use in 
quality assurance program to determine and monitor the 
precision and stability of a measurement system; a stable and 
homogenous material having physical or chemical properties, 
or both, similar to those of typical samples tested by the 
analytical measurement system. The material is properly stored 
to ensure sample integrity, and is available in sufficient quantity 
for repeated long-term testing. D6299 


3.1.14 reference material (RM), n—a material with accepted 
reference value(s), accompanied by an uncertainty at a stated 
level of confidence for desired properties, which may be used 
for calibration or quality control purposes in the laboratory. 

3.1.14.1 Discussion—Sometimes these may be prepared 
“in-house” provided the reference values are established using 
accepted standard procedures. 


3.1.15 repeatability, n—the quantitative expression of the 
random error associated with a single operator in a given 
laboratory obtaining repetitive results with the same apparatus 
under constant operating conditions on identical test material. 
It is defined as the difference between two such results at the 
95 % confidence level. D6300 


3.1.16 reproducibility, n—a quantitative expression of the 
random error associated with different operators using different 
apparatus, and so forth, each obtaining a single result on an 
identical test sample when applying the same method. It is then 
defined as the 95 % confidence limit for the difference between 
two such single and independent results. D6300 


3.1.17 site precision (R'), n—the value below which the 
absolute difference between two individual test results obtained 
under site precision conditions may be expected to occur with 
a probability of approximately 0.95 (95 %). It is defined as 2.77 
times the standard deviation of results obtained under site 
precision conditions. D6299 


3.1.18 site precision conditions, n—conditions under which 
test results are obtained by one or more operators in a single 
site location practicing the same test method on a single 
measurement system using test specimens taken at random 
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from the same sample of material over an extended period of 
time spanning at least a 15 day interval. D6299 


3.1.19 traceability, n—property of the result of a measure- 
ment or the value of a standard whereby it can be related to 
stated references, usually national or international standards, 
through an unbroken chain of comparisons all having stated 
uncertainties. 


3.2 Definitions of Terms Specific to This Standard: 

3.2.1 precision ratio (PR), n—an estimate of relative mag- 
nitude of repeatability and reproducibility. The PR for a given 
standard test method can provide information on the relative 
significance between variation caused by different operators 
and laboratories compared to a single operator in a laboratory 
performing the standard test method. 


3.2.2 test performance index (TPI), n—an approximate mea- 
sure of a laboratory’s testing capability, defined as the ratio of 
test method reproducibility (R) to site precision (R’). 


3.3 Acronyms: 
3.3.1 NIST—National Institute 
Technology, Gaithersburg, MD. 


of Standards and 


4. Significance and Use 


4.1 A petroleum products, liquid fuels, and lubricants test- 
ing laboratory plays a crucial role in product quality manage- 
ment and customer satisfaction. It is essential for a laboratory 
to provide quality data. This document provides guidance for 
establishing and maintaining a quality management system in 
a laboratory. 

4.1.1 The word ‘customer’ can refer to both customers 
internal and external to the laboratory or organization. 


5. General Quality Requirements for the Laboratory 


5.1 Establishment and maintenance of a quality manage- 
ment system shall include stated objectives in the following 
areas: a laboratory’s adherence to test method requirements, 
calibration and maintenance practices, and its quality control 
program. Laboratory quality objectives should encompass the 
laboratory’s continuous improvement goals as well as meeting 
customer requirements. 


5.2 Management shall appoint a representative to imple- 
ment and maintain the quality management system in the 
laboratory. 


5.3 Laboratory management shall review the adequacy of 
the quality management system and the activities of the 
laboratory for consistency with the stated quality objectives at 
least annually. 


5.4 The quality management system shall have documented 
processes for: 

5.4.1 Sampling and sample management (see Section 6), 

5.4.2 Data and record management (see Section 7), 

5.4.3 Control and implementation of test methods (see 
Section 8), 

5.4.4 Equipment calibration and maintenance (see Section 
9), 

5.4.5 Quality control (see Section 10), 

5.4.6 Audits and proficiency testing (see Section 11), 


5.4.7 Corrective and preventive action (see Section 13), 

5.4.8 Handling of customer complaints (see Section 14), 

5.4.9 Ensuring that procured services and materials meet the 
contracted requirements, and 

5.4.10 Ensuring that personnel are adequately trained to 
obtain quality results (see Section 15). 


6. Sampling and Sample Management 


6.1 When samples are obtained by laboratory staff, these 
samples shall be obtained in accordance with applicable 
industry standards. 


6.2 The elements of sample management shall include at a 
minimum: 

6.2.1 Procedures for unique identification of samples sub- 
mitted to the laboratory. 

6.2.2 Criteria for sample acceptance. 

6.2.3 Procedures for sample handling. 

6.2.3.1 In cases where industry standards for sample han- 
dling (for example, Practoce D5854) are applicable and refer- 
enced within industry standard test methods, they shall be 
utilized. 

6.2.4 Procedures for sample storage and retention. Items to 
consider when creating these procedures include: 

6.2.4.1 Applicable government—local, state, or national— 
regulatory requirements or customer contract agreements. 

6.2.4.2 Type of sample containers required to preserve the 
sample, 

6.2.4.3 Control of access to the retained samples to protect 
their validity and preserve their original integrity, 

6.2.4.4 Storage conditions, 

6.2.4.5 Required safety precautions, and 

6.2.4.6 Customer requirements. 

6.2.5 Procedures for sample disposal in accordance with 
applicable government regulatory requirements. 


Note 2—This may be handled through a separate chemical hygiene or 
waste disposal plan. 


7. Data and Record Management 


7.1 Reports of Analysis: 

7.1.1 The work carried out by a laboratory shall be covered 
by a certificate or report that accurately and unambiguously 
presents the test results and all other relevant information. 


Note 3—This report may be an entry in a Laboratory Information 
Management System (LIMS) or equivalent system. 

7.1.2 The following items are suggested for inclusion in 
laboratory reports: 

7.1.2.1 Name and address of the testing laboratory, 

7.1.2.2 Unique identification of the report (such as serial 
number) on each page of the report including version identifi- 
cation if the report has been updated, 


Note 4—Occasionally, a report may be updated and a version identi- 
fication will enable one version of the report to be distinguished from 
another. This is necessary to determine which report version was the 
original and which is the most current. Simple conventions such as last 
updated date are useful means of version identification. 


7.1.2.3 Name and address of the customer, 
7.1.2.4 Order number, 
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7.1.2.5 Description and identification of the test sample 
including comments on the sample condition particularly if it is 
likely to have an adverse effect on the sample integrity, 

7.1.2.6 Date of receipt of the test sample and date(s) of 
performance of test, as appropriate, 

7.1.2.7 Identification of the test specification, method, and 
procedure, 

7.1.2.8 Description of the sampling procedure, where 
relevant, 

7.1.2.9 Any deviations, additions to or exclusions from the 
specified test requirements, and any other information relevant 
to a specific test, 

7.1.2.10 Disclosure of any nonstandard test method or 
procedure utilized, 

7.1.2.11 Measurements, examinations, and derived results 
including units of measurement, supported by tables, graphs, 
sketches, and photographs as appropriate, and any failures 
identified, 

7.1.2.12 Minimum-maximum product specifications, if 
applicable, 

7.1.2.13 A statement of the measurement uncertainty (where 
relevant or required by the customer), 

7.1.2.14 Any other information which might be required by 
the customer, 

7.1.2.15 A signature and job title of person(s) accepting 
technical responsibility for the test report and the date of issue, 
and 

7.1.2.16 A statement on the laboratory policy regarding the 
reproduction of test reports. 

7.1.3 Items actually included in laboratory reports should be 
specified by laboratory management or agreements with 
customers, or both. 

7.1.4 Procedures for corrections or additions to a test report 
after issue shall be established. 


7.2 Reporting and Rounding the Data: 

7.2.1 The reporting requirements specified in the test 
method or procedure shall be used (unless specifically required 
otherwise by the customer or applicable regulations). 

7.2.2 If rounding is performed, the rounding protocol of 
Practice E29 should be used unless otherwise specified in the 
method, procedure, or governing specification. 


7.3 Records of Calibration and Maintenance: 

7.3.1 Procedures shall be established for the management of 
instrument calibration records. Such records usually indicate 
the instrument calibrated, method or procedure used for 
calibration, the dates of last and next calibrations, the person 
performing the calibration, the values obtained during 
calibration, permissible tolerances, and the nature and trace- 
ability (if applicable) of the calibration standards (that is, 
certified values). Records may be electronic. 

7.3.2 Procedures shall be established for the management of 
instrument maintenance records. Such records usually indicate 
the instrument maintained, description of the maintenance 
performed, the dates of last and next maintenance, and the 
person performing the maintenance. Records may be elec- 
tronic. 


Note 5—For instruments that require calibration, calibration and 


maintenance records may be combined. 


7.4 Quality Control (QC) Testing Records: 

7.4.1 The laboratory shall have documented procedures for 
creating and maintaining records for analysis of QC samples. It 
is recommended that such records include the sample name and 
source, the test(s) for which it is to be used, the assigned values 
and their uncertainty where applicable, and values obtained 
upon analysis. Additionally, it is recommended that the receipt 
date or date put into active QC use in the laboratory be 
documented, along with the expiration date (if applicable). 

7.4.2 Procedures for retaining completed control charts 
should be established. It is recommended that these records 
include the date the control charts were changed and the reason 
for the change. 


7.5 Record Retention: 

7.5.1 The record system should suit the laboratory’s particu- 
lar circumstances and comply with any existing regulations and 
customer requirements. 

7.5.2 All data shall be maintained according to laboratory, 
company, customer, or regulatory agency requirements, or a 
combination thereof. 

7.5.3 Procedures for retaining records, including electronic, 
of all original observations, calculations and derived data, 
calibration records, and final test reports for an appropriate 
period shall be established. The records for each test should 
contain sufficient information to permit satisfactory replication 
of the test and recalculation of the results. 

7.5.4 The records shall be held in a safe and secure storage. 
A system shall exist that allows locating the required docu- 
ments in a reasonable period of time. 


7.6 Document Control—A document control process shall 
be established. As a minimum, for laboratory procedures (for 
example, standard operating procedures, SOPs) and laboratory 
test procedures (if used), a document control process shall 
address version/revision identification and management ap- 
proval. 


8. Test Methods 


8.1 The laboratory shall have documented test methods and 
procedures for performing the required tests. 


8.2 The test methods that are stated in the product specifi- 
cations or agreed upon with customers shall be used for sample 
analysis. 


8.3 These test methods shall be maintained up-to-date and 
be readily available to the laboratory staff. 

Note 6—Some specifications or regulations may require the use of a 
specific year version of a test method. 

8.4 The laboratory shall have procedures for the approval, 


documentation, and reporting of deviations from the test 
method requirements or the use of alternative methods. 


9. Equipment Calibration and Maintenance 

9.1 Calibration: 

9.1.1 Procedures shall be established to verify the measur- 
ing and testing equipment in-calibration status, at a scheduled 
frequency. 
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9.1.2 Items to consider when creating these procedures 
include: 
9.1.2.1 Records of calibration and maintenance (see 7.3), 


Note 7—The calibration verification frequency and protocol may vary 
with the instrument type and its frequency of use. 


9.1.3 Traceability to national or international standards, 


Note 8—Where the concept of traceability to national or international 
standards of measurement is not applicable, the testing laboratory shall 
provide satisfactory evidence of test result accuracy (for example, by 
participation in a program of interlaboratory comparisons). 

9.1.4 Requirements of the test method or procedure, 

9.1.5 Customer requirements, and 

9.1.6 Corrective actions (see Section 13). 


9.2 Maintenance: 

9.2.1 Laboratories shall have procedures to ensure that 
measuring and testing equipment is properly maintained, 

9.2.2 An inventory of measuring and testing equipment is 
established and maintained, 

9.2.3 A reliability strategy is developed which may include 
the following: 

9.2.3.1 Equipment age, 

9.2.3.2 Back-up equipment 
utilization, 

9.2.3.3 Critical spare parts, and 

9.2.4 A maintenance schedule is established and followed. 


justification based on 


9.3 The performance of apparatus and equipment used in 
the laboratory but not calibrated in that laboratory (that is, 
pre-calibrated, vendor supplied) should be verified by using a 
documented, technically valid procedure at periodic intervals. 


9.4 Calibration standards shall be appropriate for the 
method and characterized with the accuracy demanded by the 
analysis to be performed. Quantitative calibration standards 
should be prepared from constituents of known purity. Use the 
primary calibration standards or CRMs specified or allowed in 
the test method. 

9.4.1 Where appropriate, values for reference materials 
should be produced following the certification protocol used by 
NIST?!®!! or other standards issuing bodies, and, should be 
traceable to national or international standard reference 
materials, if required or appropriate. 

9.4.2 The materials analyzed in proficiency testing pro- 
grams meeting the requirements of Practice D6300 or ISO 
4259 may be used as reference materials, provided no obvious 
bias or unusual frequency distribution of results are observed. 
The consensus value is most likely the value closest to the true 
value of this material; however, the uncertainty attached to this 
mean value will be dependent on the precision and the total 
number of the participating laboratories. 


9.5 The laboratory shall establish procedures for the storage 
of reference materials in a manner to ensure their safety, 
integrity, and protection from contamination (see 6.2.4). 


° Cali, J. P., Anal. Chem., Vol 48, 802A, 1976. 

10 Uriano, G. A., and Gravatt, C. C., CRC Crit. Revs, in Anal. Chem., Vol 6, 361, 
1977. 

1 Alvarez, R., Rasberry, S. D., and Uriano, G. A., Anal. Chem., Vol 54, 1226A, 
1982. 


9.6 Records of instrument calibration shall be maintained 
(see Section 7). 


9.7 If an instrument is found to be out of calibration, and the 
situation cannot be immediately addressed, then the instrument 
shall be taken out of operation and tagged as such until the 
situation is corrected (see Section 13). 


10. Quality Control 


10.1 Quality Control Practices: 

10.1.1 Quality control practices shall be established to 
assess applicable test methods used by the laboratory. 

10.1.2 Use of appropriate quality control charts or other 
quality control practices shall be established for each test 
method performed by the laboratory unless specifically ex- 
cluded (see Practice D6299). Document cases where quality 
control practices are not employed and include the rationale. 

10.1.3 This practice advocates the regular testing of quality 
control samples with timely interpretation of test results. This 
practice also advocates using appropriate control charting 
techniques to ascertain the in-statistical-control status of test 
methods in terms of precision, bias (if a standard is being 
used), and method stability over time. For details concerning 
QC sample requirements and control charting techniques, refer 
to Practice D6299. The generally accepted practices are out- 
lined in 10.1.4 through 10.4.4. 

10.1.4 QC sample testing frequency shall be established for 
each applicable test method. Principal factors to be considered 
for determining the frequency of testing shall include: (/) 
frequency of use of the analytical measurement system, (2) 
criticality of the parameter being measured and business 
economics, (3) established system stability and precision 
performance based on historical data, (4) regulatory 
requirements, (5) contractual provisions, and (6) test method 
requirements. Minimum QC sample testing specified in regu- 
lations or in the test method shall prevail over any larger 
interval determined below in 10.1.4.1 — 10.1.4.4. 

10.1.4.1 If site precision for a specific test has not been 
established as defined by Practice D6299, then the recom- 
mended frequency for analysis of QC samples is one QC out of 
every ten samples analyzed. Alternatively, one QC sample is 
analyzed each day that samples are analyzed, whichever is 
more frequent. 

10.1.4.2 Once the site precision has been established as 
defined by Practice D6299, and to ensure similar quality of 
data is achieved with the documented method, the minimal QC 
frequency may be adjusted based on the Test Performance 
Index (TPI) and the Precision Ratio (PR). 

10.1.4.3 Table 1 provides recommended minimal QC fre- 
quencies as a function of PR and TPI. For those tests, which are 
performed infrequently, for example less than 25 samples are 
analyzed monthly, it is recommended that at least one QC 
sample be analyzed each time samples are analyzed. 

10.1.4.4 In many situations, the minimal QC frequency as 
recommended by Table 1 may not be sufficient to ensure 
adequate statistical quality control, considering, for example, 
the significance of use of the results. Hence, it is recommended 
that the flowchart in Fig. 1 be followed to determine if a higher 
QC frequency should be used. 
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TABLE 1 Minimal QC Frequency as a Function of 
Test Performance Index 


TPI for TPI for Nominal QC Approximate 
Standard Standard Frequency Percentage 
Test Methods Test Methods (1 QC out of every of QC Samples/ 
with PR<4 with PR=4 X Samples) Total Analyses 

Values of X 
Not Not 10 9 
determined determined 
<0.8 <1.6 10 9 
0.8-1.2 1.6-2.4 20 5 
1.2-2.0 2.4—4.0 35 3 
>2.0 >4.0 40 2 


10.1.4.5 The TPI should be recalculated and reviewed at 
least annually. Adjustments to QC frequency should be made 
based on the recalculated TPI by following sections 10.1.4.1 
and 10.1.4.2. 

10.1.5 QC testing frequency, QC samples, and their test 
values shall be recorded. 

10.1.6 All persons who routinely operate the system shall 
participate in generating QC test data. QC samples should be 
treated as regular samples. 


Note 9—Avoid special treatment of QC samples designed to “get a 
better result.” Special treatment seriously undermines the integrity of 
precision and bias estimates. 

10.1.7 The laboratory may establish random or blind 
testing, or both, of QC or other known materials. 


10.2 Quality Control Sample and Test Data Evaluation: 

10.2.1 QC samples should be stable and homogeneous 
materials having physical or chemical properties, or both, 
representative of the actual samples being analyzed by the test 
method. This material shall be well-characterized for the 
analyses of interest, available in sufficient quantities, have 
concentration values that are within the calibration range of the 
test method, and reflect the most common values tested by the 
laboratory. For QC testing that is strictly for monitoring the test 
method stability and precision, the QC sample expected value 
is the control chart centerline, established using data obtained 
under site precision conditions. For regular QC testing that is 
intended to assess test method bias, RMs, or CRMs with 
independently assigned ARVs should be used. The results 
should be assessed in accordance with Practice D6299 require- 
ments for check standard testing. For infrequent QC testing for 
bias assessment, refer to Practice D6617. 


Norte 10—It is not advisable to use the same sample for both a calibrant 
and a QC sample. It is not advisable to use the same chemical lot number 
for both a calibrant and a QC sample. 

10.2.2 If the QC material is observed to be degrading or 
changing in physical or chemical characteristics, this shall be 
immediately investigated and, if necessary, a replacement QC 
material shall be prepared for use. 


Norte 11—In a customer-supplier quality dispute, it may be beneficial to 
provide the customer with the laboratory’s test results on QC material to 
demonstrate testing proficiency. Practice D3244 may be useful. 

10.3 Quality Control Charts: 

10.3.1 QC sample test data shall be promptly plotted on a 
control chart and evaluated to determine if the results obtained 
are within the method specifications and laboratory-established 


control limits. The charts used should be appropriate for the 
testing conditions and statistical objectives. Corrective action 
should be taken and documented for any analyses that are 
out-of-statistical-control (see Section 13). 


Note 12—Charts such as individual, moving average and moving 
range, exponentially weighted moving average, or cumulative summation 
charts may be used as appropriate. Refer to Practice D6299 for guidance 
on plotting these charts. 

10.3.1.1 The charts should indicate the test method, date 
when the QC analyses were performed, and who performed 
them. Test samples should not be analyzed or results for 
samples should not be reported until the corresponding QC 
data are assessed and the testing process is verified to be 
in-statistical-control. (See 10.1.) 

10.3.2 Adequate training shall be provided to the organiza- 
tional entity that is responsible for the generation and interpre- 
tation of control charts. 

10.3.3 It is suggested that the charts be displayed promi- 
nently near the analysis workstation or be readily retrievable 
from a computer workstation, so that all can view and, if 
necessary, help in improving the analyses. 

10.3.4 Supervisory and technical personnel assigned this 
task shall periodically review the QC charts. 

10.3.5 The laboratory shall establish written procedures 
outlining the appropriate interpretation of QC charts and 
responses to out-of-statistical-control situations observed. 

10.3.5.1 When an out-of-statistical-control situation has 
been identified, an investigation shall be conducted and reme- 
dial action (if needed) should be taken before analyzing further 
samples. In all such cases, document any actions taken and 
retest the QC sample and ensure that a satisfactory result can be 
obtained before analyzing unknown samples. 


Note 13—A generic checklist for investigating the root cause of 
unsatisfactory analytical performance is given in Appendix X1. 

10.3.6 Out-of-statistical-control situations may be detected 
by one or more analyses. In these situations, procedures shall 
be documented to determine when it may be necessary to retest 
samples analyzed during the period between the last in- 
statistical-control QC data point and the QC data point that 
triggered the out-of-statistical-control notice (or event) using 
retained samples and equipment known to be in control. If the 
new analysis shows a difference that is statistically different 
from the original results, and the difference exceeds the 
established site precision of that test, the laboratory shall 
decide on what further actions are necessary (see Section 13). 


10.4 Revision of Control Charts—QC chart revision is 
covered in detail in Practice D6299. Control charts shall be 
revised only when the existing limits are no longer appropriate. 
As a guideline, revisions may be needed when: 

10.4.1 Additional information becomes available, 

10.4.2 The process has improved, 

10.4.3 A new QC material is initiated and the mean value is 
different than the previous QC material, or 

10.4.4 There are major changes to the test procedure. 


11. Audits and Proficiency Testing 
11.1 Audits: 
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What is the minimal 
frequency to analyze 
a QC sample for a 
STM? 


Has the TPI Refer to Table 1 for the 
been recommended minimal 
determined QC frequency based on 
for this the laboratory's TPI and 
STM? STM’s PR 


Periodically recalculate 
and review TPI 


Recommend analyzing Would a QC 


the QC sample at a sample be 

frequency of one QC analyzed at 
sample out of every 10 least once a 
samples or 1 QC sample month? 


should be analyzed each 
day that samples are 
analyzed, whichever is 

more frequent 


If the STM is 
out-of-statistical- Recommend analyzing 
control, the QC sample at a 
would there be frequency greater than 
a significant the value listed in Table 1 
economic 


penalty? 


Are there 
regulatory or 
Contractual provision 
which would require the 
analysis of greater 


Recommend analyzing the 
QC sample at a frequency 
equal to or greater than the 
minimal frequency listed in 
Table 1 


FIG. 1 Flowchart for QC Frequency 
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11.1.1 A laboratory shall have a system to periodically 
review its own practices to confirm continued conformance to 
the laboratory’s documented quality management system. 
Even if the laboratory is subjected to a formal external audit 
(for example, as a requirement of ANSI/ISO/ASQ Q9000), it is 
important to have internal audits since the internal reviewers 
may be more familiar with their laboratory’s requirements than 
the external auditors. An assessment checklist based on this 
standard is provided in Appendix X2. 

11.1.2 Audits of test methods (sometimes referred to as test 
method assessments) shall be conducted to confirm adherence 
to the documented test methods and test method procedures. 
The performance of the entire test should be observed and 
checked against the official specified test method. The fre- 
quency of test method assessments should be specified in the 
quality management system; an annual audit of test methods is 
recommended. 


Note 14—These audits may be part of the quality management system 
audits or may be separate. 

11.1.3 Audit results shall be promptly documented. The 
team shall report the audit results to laboratory personnel 
having the authority and responsibility to take corrective action 
and to its management. 

11.1.4 The findings and recommendations of these internal 
audits shall be reviewed by the laboratory management and 
acted upon to correct the deficiencies or nonconformances. 

11.1.5 The effectiveness of any corrective actions taken in 
response to an audit shall be verified. The follow-up results 
shall be documented as required by the quality management 
system procedures or laboratory policy, or both. 


11.2 Proficiency Testing: 

11.2.1 Regular participation in interlaboratory proficiency 
testing (PT) programs, where appropriate samples are tested by 
multiple test facilities using a specified test protocol, shall be 
integrated into the laboratory’s quality control program. Pro- 
ficiency test programs shall be used as appropriate by the 
laboratory to demonstrate testing proficiency relative to other 
industry laboratories. 


Note 15—Document the rationale for not participating in a proficiency 
test program. 

11.2.2 The laboratory shall establish criteria for guiding 
their participation in interlaboratory testing programs. Such 
criteria may include factors such as the frequency of use of the 
target test method, the critical nature of how the customer uses 
the data, and regulatory considerations. Participation in profi- 
ciency test programs can provide a cost-effective alternative to 
regular CRM testing. 

11.2.3 Participants may plot their deviations from the con- 
sensus values established by the proficiency test program 
averages on a control chart to ascertain if their measurement 
processes are non-biased. The precision of these exchange 
performance data can also be assessed against precision 
established by in-house QC sample testing for consistency (see 
Practice D6299 for details). 

11.2.4 Additional guidance related to the analysis and inter- 
pretation of proficiency test program results is provided in 
Guide D7372. 


11.2.5 Participation in proficiency testing shall not be con- 
sidered as a substitute for a quality control program, as 
described in 10.1, and vice versa. 


12. Test Method Precision Performance Assessment 


12.1 The test performance index (TPI) can be used to 
compare the precision of the laboratory measurements with the 
published reproducibility of a standard test method. The term 
TPI is defined as: 


test method reproducibility 


test performance index = z _ (1) 
site precision 


Note 16—The ASTM International Committee D02 sponsored Inter- 
laboratory Crosscheck Program employs a test performance index based 
on the ratio of the published ASTM reproducibility to the reproducibility 
calculated from the program data. This index is termed the TPI (Industry) 
to distinguish from the definition in 12.1. 


12.2 A precision ratio (PR) is determined for a given 
published test method so that the appropriate action criteria 
may be applied for a laboratory’s TPI. The PR for a published 
test method estimates the influence that non-site specific 
variations has on the published precision. The PR can be 
calculated by dividing the test method’s reproducibility by the 
repeatability as shown in Eq 2. 


Test Method reproducibility (R) 
Test Method repeatability (r) 


Precision Ratio, PR = (2) 
where the ratio of R/r is calculated to the nearest integer (that 
is, 1, 2, 3, 4, ...). 

12.2.1 A test method with PR greater than or equal to 4, for 
the purpose of this practice, is deemed to exhibit a significant 
difference between repeatability and reproducibility. For fur- 
ther explanation on why the greater than or equal to 4 criterion 
was chosen, please see Appendix X3. 


12.3 A laboratory’s TPI may be a function of the sample 
type being analyzed and variations associated with that labo- 
ratory. As general guidelines Table 2 may be used once the TPI 
of that laboratory and the PR of the published standard test 
method has been calculated. Similar information to that pro- 
vided in Table 2 is provided in 12.3.1 through 12.3.2.3. 

12.3.1 For a published standard test method with a PR less 
than 4 the following TPI criteria should be applied. 

12.3.1.1 A TPI greater than 1.2 indicates that the perfor- 
mance is probably satisfactory relative to ASTM published 
precision. 

12.3.1.2 A TPI greater than or equal to 0.8 and less than or 
equal to 1.2 indicated performance may be marginal and the 
laboratory should consider method review for improvement. 

12.3.1.3 A TPI less than 0.8 suggests that the method as 
practiced at this site is not consistent with the ASTM published 
precision. Either laboratory method performance improvement 
is required, or ASTM published precision does not reflect 
achievable precision. Existing interlaboratory exchange perfor- 
mance (if available) should be reviewed to determine if the 
latter is plausible. 

12.3.2 For a published standard test method with a PR 
greater than or equal to 4 the following TPI criteria should be 
applied. 
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TABLE 2 Guidelines for Action Based on TPI 


TPI for Standard 
Test Methods 


TPI for Standard 
Test Methods 


Recommended Quality 
Improvement Action 


with PR<4 with PR=4 
>1.2 >2.4 Indicates that the performance is probably 
satisfactory relative to ASTM published precision. 
>0.8 and <1.2 >1.6 and <2.4 Indicates that the performance is probably 
satisfactory relative to ASTM published precision, 
however a method review could be necessary to 
improve its performance. 
<0.8 <1.6 This condition suggests that the method as 


12.3.2.1 A TPI greater than 2.4 indicates that the perfor- 
mance is probably satisfactory relative to ASTM published 
precision. 

12.3.2.2 A TPI greater than or equal to 1.6 and less than or 
equal to 2.4 indicated performance may be marginal and the 
laboratory should consider method review for improvement. 

12.3.2.3 A TPI less than 1.6 suggests that the method as 
practiced at this site is not consistent with the ASTM published 
precision. Either laboratory method performance improvement 
is required, or ASTM published precision does not reflect 
precision achievable. Existing interlaboratory exchange perfor- 
mance (if available) should be reviewed to determine if the 
latter is plausible. 

12.3.3 A laboratory may choose to set other benchmarks for 
TPI, keeping in mind that site precision of an adequately 
performing laboratory cannot, in the long run, exceed the 
practically achievable reproducibility of the method when PR 
is less than 4 or approaches repeatability when PR is much 
greater than 4. 


Note 17—Experience has shown, for some methods, published repro- 
ducibility is not in good agreement with the precision achieved by 
participants in well-managed crosscheck programs. Users should consider 
this fact when evaluating laboratory performance using TPI. 

12.4 A laboratory should review their precision obtained for 
multiple analyses on the same sample. The site precision of the 
QC samples can be compared with the reproducibility or 
repeatability given in the standard test methods to indicate how 
well a laboratory is performing against the industry standards. 


12.5 A laboratory’s site precision (R') that is significantly 
worse than the published test method reproducibility may 
indicate poor performance. An investigation should be 
launched to determine the root cause for this performance so 
that corrective action can be undertaken if necessary. Such a 
periodic review is a key feature of a laboratory’s continuous 
improvement program. 


13. Corrective and Preventive Action 


13.1 A corrective and preventive action process shall be 
established. The need for corrective and preventive action may 
be indicated by one or more of the following unacceptable 
situations: 


practiced at this site is not consistent with the 
ASTM published precision. Either laboratory 
method performance improvement is required, or 
the ASTM published precision does not reflect 
precision achievable. Existing interlaboratory 
exchange performance (if available) should be 
reviewed to determine if the latter is plausible. 


13.1.1 Equipment out of calibration, 

13.1.2 QC or check sample result out-of-statistical-control, 

13.1.3 Test method performance by the laboratory does not 
meet performance criteria (for example, precision, bias, and the 
like) documented in the test method, 

13.1.4 Product, material, or process out of specification 
data, 

13.1.5 Outlier or unacceptable trend in an interlaboratory 
cross-check program, 

13.1.6 Nonconformance identified in an external or internal 
audit, 

13.1.7 Nonconformance identified during review of labora- 
tory data or records, 

13.1.8 Customer complaint. 


13.2 When any of these situations occur, the root cause 
should be investigated and identified. Procedures for investi- 
gating root cause should be established. Items to consider when 
creating these procedures include: 

13.2.1 Determining when the test of equipment was last 
known to be in control, 

13.2.2 Identifying results that may have been adversely 
affected, 

13.2.3 How to handle affected results already reported to a 
customer, 

13.2.4 What to do if the root cause cannot be determined, 
and 

13.2.5 What to do if it is determined that the original data is 
correct. 

13.2.6 It is possible that the analytical results are correct, 
even if they don’t meet specifications. Procedures should 
consider this possibility. 

13.2.7 See Appendix X1 for a checklist for investigating the 
root cause of unsatisfactory analytical performance. 


13.3 Procedures should also be established for the identi- 
fication and implementation of appropriate corrective and 
preventive action so that the situation does not reoccur. This 
may involve: 

13.3.1 Training or retraining personnel, 

13.3.2 Reviewing customer specifications, 

13.3.3 Reviewing test methods and procedures, 
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13.3.4 Establishing new or revised procedures, 

13.3.5 Instrument maintenance and repair, 

13.3.6 Re-preparation of reagents and standards, 

13.3.7 Recalibration of equipment, 

13.3.8 Re-analysis of samples, and 

13.3.9 Additional QC sample analysis. 

13.3.10 The situation, root cause, and corrective/preventive 
action taken should be documented promptly. A corrective and 
preventive action report is a suitable format for documentation. 

13.3.11 The report should be reviewed and approved by 
management and then verified for effectiveness of corrective 
and preventive actions. 


13.4 Quality control charts (see 10.3) are a method of 
preventive action and should be evaluated on a regular basis to 
prevent, when possible, out-of-statistical-control situations. 


14. Customer Complaints 


14.1 A procedure shall exist to follow-up on customer 
complaints or non-conformances brought to the laboratory’ s 
attention by a client. The result of such investigation should be 
communicated to the customer as soon as practical. This 
procedure may be incorporated into a corrective and preventive 
action process (see Section 13). 


15. Training 


15.1 Laboratory management shall ensure that all staff 
performing testing or interpreting data, or both, are appropri- 
ately trained. 


15.2 Laboratory training should cover at a minimum the 
following areas: safety, test methods, and company policies 
and procedures. Training is specifically required as specified 
in: 5.4.10, 10.3.2, 13.3.1, and X1.1.11. 


15.3 Records of training and the determination of compe- 
tence shall be maintained. 


16. Relationship with Other Quality Standards 


16.1 Some laboratories in the petrochemicals testing area 
have been registered to ISO/IEC 17025. There are a number of 


similarities between the ISO standard and this practice in key 
areas of managing laboratory quality. For example: 


Requirement ISO/IEC ASTM 
17025 Practice D6792 
Quality Management System 4.2 5.1 
Document Control 4.3 7.6; 8.1; 8.4 
Contract Review 4.4 5.4.9 
Complaints 4.8 14.1 
Corrective Action 4.10 13; Appendix X1 
Preventive Action 4.11 13.4 
Control of Records 4.12 7.3.1; 7.4; 7.5; 7:6 
Internal Audits 4.13 WA 
Management Reviews 4.14 5.3 
Personnel 5.2 5.4.10; 15.1, 15:2 
Calibration 5.6.2.1 92-97 
Sample Handling 5.8 6.2 
Quality Control Procedures 5.9 10.1 
Use of Quality Control Materials 5.9.a 10.2 
Proficiency Testing 5.9.b 11.2 
Data Reports 5.10 TA 


16.2 Measurement Uncertainty—For test methods under the 
jurisdiction of Committee D02, measurement uncertainty as 
required in ISO/IEC 17025, as practiced by a laboratory, can be 
estimated by multiplying 2x the site precision standard devia- 
tion as defined in Practice D6299. 


Nore 18—The complexity and empirical nature of the majority of D02 
methods preclude the application of rigorous measurement uncertainty 
algorithms. In many cases, interactions between the test method variables 
and the measurand cannot be reasonably estimated due to the covariance 
of the variables that affect the measurand. The site precision approach 
estimates the combined effects of these variables on the total uncertainty 
for the measurand. 


16.3 The practice of using QC materials and control charts 
to estimate measurement uncertainty assumes that the labora- 
tory bias is not statistically or practically significant. This 
assumption should be validated periodically using check stan- 
dards. See Practice D6617 or Practice D6299 for further 
guidance. 


17. Keywords 


17.1 audit; calibration; control charts; proficiency testing; 
quality assurance; quality control; test performance index 


APPENDIXES 


(Nonmandatory Information) 


X1. CHECKLIST FOR INVESTIGATING THE ROOT CAUSE OF UNSATISFACTORY ANALYTICAL PERFORMANCE 


X1.1 To identify why a laboratory’s data may have been 
considered a statistical outlier or to improve the precision, or 
both, the following action items (not necessarily in the order of 
preference) are suggested. There may be additional ways to 
improve the performance. 


X1.1.1 Check the results for typos, calculation errors, and 
transcription errors. 


X1.1.2 Reanalyze the sample; compare results to site 
precision, or, if not available, to test method repeatability. 
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X1.1.3 Check the sample for homogeneity or 
contamination, and that a representative sample has been 
analyzed. 

X1.1.4 Review the test method and ensure that the latest 
version of the ASTM test method is being used. Check the 
procedure step-by-step with the analyst. 


X1.1.5 Check the instrument calibration. 


X1.1.6 Check the statistical quality control chart to see if the 
problem has been developing earlier. 
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X1.1.7 Check the quality of the reagents and standards used, 
and whether they are expired or contaminated. 


X1.1.8 Check the equipment for proper operation against 
the vendor’s operating manual. 


X1.1.9 Perform maintenance or repairs, or both, on the 
equipment following guidelines established by the vendor. 


X1.1.10 After the problem has been resolved, analyze a 
certified reference material if one is available, or the laboratory 
quality control sample, to ascertain that the analytical operation 
is under control. 


X1.1.11 Provide training to new analysts and, if necessary, 
refresher training to experienced analysts. 


X1.1.12 Document the incident and the learnings for use in 
the future if a similar problem occurs. 


X2. ASSESSMENT CHECKLIST TO EVALUATE COMPLIANCE WITH PRACTICE D6792 


X2.1 See the checklist in Table X2.1. The requirements 
covered in D6792 are summarized in checklist Table X2.1. For 
most items in the checklist, a reference to the corresponding 
text in D6792 is included in (). Note that requirements listed in 
the checklist with an asterisk (*) are mandatory (“shall”). See 
X2.3 for a discussion on possible rating schemes. To make the 
best possible use of this checklist, it is recommended that the 
assessor provide comments for each item. The comments 
should include a description of the documents, data and 
activities observed and should indicate any gaps, omissions, or 
incorrect actions. 


X2.2 Table X2.2 presents the actions and/or questions that 
the assessor could use to supplement their use of the checklist 
(Table X2.1). This table is intended as a general guide to 
provide additional information to assist the auditor in complet- 
ing the Checklist. This table is not intended to be all inclusive 
nor to have every item apply to all petroleum, petroleum 
product, liquid fuel, and lubricant laboratories. Auditors are 
expected to use their professional judgement in applying this 
supplemental guide. 


X2.3 Rating Schemes 


X2.3.1 General—Professional auditors often use rating 
schemes developed by their respective organization, so there a 
number of ways to approach developing an overall rating for a 
laboratory based on the Table X2.1 checklist. Two generalized 
schemes are discussed below, without any recommendation of 
one over the other. 


X2.3.2 Non-Numeric Rating—Some audits/assessments are 
conducted with the sole purpose of demonstrating compliance. 
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The purpose of such audits could be for a laboratory manager 
to identify gaps in their efforts to satisfy D6792 or the audit 
could be conducted to satisfy a regulatory compliance. In both 
cases, the identification of all gaps, omissions or incorrect 
activities are important to note. The auditor would use profes- 
sional judgement and the relative seriousness of any gaps (for 
example, did it involve a mandatory requirement) and the 
frequency of occurrence (often versus a one-off event) to rank 
the findings. The objective of the laboratory following an audit 
would be close all gaps giving priority to the more significant 
findings. 


X2.3.3 Numeric Rating Schemes—Numeric scores might be 
useful with a network of laboratories or in cases where 
quantifiable means are needed to show progress from one audit 
to another. In general numeric rating schemes involve assign- 
ing points to each checklist item based on how completely a 
requirement is satisfied. For example, one could develop a 
scoring profile by assigning a rating number to indicate that the 
laboratory fully met all the requirements of an item and then a 
lower numbers to indicate partial completion or satisfaction of 
the requirement with the lowest rating of O (zero) indicating 
that the requirement was not met. In cases where this type of 
scheme is used the numeric rating can be weighted by the 
relative importance of a requirement or group of requirements. 
For example, one could adopt a weighted scheme as demon- 
strated in Table X2.3. The final audit score would result from 
the use of this or a similar table. The decision or guidance on 
whether or not a given final score was sufficient to meet the 
needs of the auditing organization or the laboratory manage- 
ment would also need to be developed. 
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TABLE X2.1 Assessment Checklist to Evaluate Compliance with Practice 


Requirements Rating | Assessor Comments 


QUALITY MANAGEMENT SYSTEM (QMS) 


*A quality management system is established and maintained. (5.1) 


-ajajat 
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*Management has appointed a representative to implement and maintain the quality management system 
in the laboratory. (5.2) 


*The quality management system includes stated objectives in the following areas: (5.1) 
e Laboratory’s adherence to test method requirements, 

e Calibration and maintenance practices, 

e Quality control program, 

e Continuous improvement goals, 

e Meeting customer requirements. 


*The quality management system has documented processes for: (5.4) 

Sample management, 

e Data and record management, 

e Producing accurate, reliable, and properly represented test results, 

e Audits and proficiency testing, 

e Test Method, 

e Corrective and preventive action, 

e Ensuring that procured services and materials meet the contracted requirements, 
e Ensuring that personnel are adequately trained to obtain quality results 


*Laboratory management reviews the adequacy of the quality management system and the activities 
of the laboratory for consistency with the stated quality objectives at least annually. (5.3) 


TEST METHODS 


*Laboratory has documented, up-to-date and readily available (to the laboratory staff) test methods and 
procedures for performing the required tests. (8.1) 


*Test methods used for customer sample analyses are those stated in respective product specifications, 
regulations, or are those agreed upon with the customer. (8.1) 


*Laboratory has documented procedures for the approval, documentation, and reporting of deviations 
from the published test method requirements. (8.2) 


*Laboratory has documented procedures for obtaining customer approval for the use of alternative or 
substitute test methods for product certification. (8.2) (See Practice D6708 for applicable procedures.) 


SAMPLING AND SAMPLE MANAGEMENT 


*A sample management process or system is established and maintained. (6.1) 


*Sample management system addresses procedures for unique sample identification and criteria 
for sample acceptance. (6.2.1, 6.2.2) 


*Sample management system addresses procedures for sample handling, retention and storage. The 
system addresses the following items when applicable: (6.2.3, 6.2.4) 
Government (local, state, or national) regulatory requirements for shelf life, 

e Data or information regarding sample stability with respect to tested parameters, 

e Type of sample containers required to preserve the sample, 

e Sample storage conditions before testing, 

e Sample storage conditions for sample retention, 

e Required safety precautions, 

e Customer requirements, 

e Control of access to the retained samples to protect their validity and preserve their original integrity. 


*Procedures for sample disposal are in accordance with applicable regulatory requirements. (This may be 
handled through a separate chemical hygiene or waste disposal plan.) (6.2.5) 


CALIBRATION 


A laboratory calibration system or program is established. (5.1) 

The calibration system addresses: (9.1) 

e Creation and use of calibration records, 

e Lab calibration schedule, 

e Corrective actions. 

Calibration procedures are established for each instrument or test method as appropriate. (9.1) 

The calibration procedures address the following: 

e Calibration schedules, (The calibration frequency may vary with the instrument type and its frequency of 
use, some needing calibration before each set of analyses, others requiring calibration at less frequent peri- 
ods or triggered by a QC chart out-of-statistical-control situation.) 


e Special requirements of the test method or procedure, 
e Customer requirements. 


The performance of apparatus and equipment used in, but not calibrated in the laboratory (that is, pre- 
calibrated, vendor supplied), is verified using a documented, technically valid procedure at periodic inter- 
vals. (9.2) 
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TABLE X2.1 Continued 


4.4 


*Calibration standards meet the following requirements or conditions when appropriate: (9.3) 

e Are appropriate for the method and are characterized with the accuracy demanded by the analysis to 
be performed, 

e Are traceable to national or international standards or are derived from constituents of known purity, 
when prepared by the laboratory, 

e Where the concept of traceability to national or international standards of measurement is not 
applicable, the testing laboratory provides, satisfactory evidence of test result accuracy, for example, by 
participation in a program of interlaboratory comparisons, (Note 8) 

e Primary calibration standards or CRMs are used when specified or allowed by the test method, 

e Where appropriate, values for reference materials are produced following appropriate (NIST or other 
standards issuing body) certification protocol, (9.4.1) 

e The materials analyzed in proficiency testing programs meeting the requirements of Practice D6300 or 
ISO 4259 are used as reference materials, provided no obvious bias or unusual frequency distribution of 
results are observed. (The consensus value is most likely the value closest to the true value of this mate- 
rial; however, the uncertainty attached to this mean value will be dependent on the precision and the total 
number of the participating laboratories.) (9.4.2) 


4.5 


*The laboratory has procedures for the storage of calibration and reference materials in a manner to en- 
sure their safety, integrity, and protection from contamination (see 6.2.4). (9.5) 


4.6 


*Calibration records are maintained. (9.6) 


4.7 


Procedures are established for the management of instrument calibration records. (7.3.1) 
Such records indicate the following when applicable: 

e Instrument calibrated, 

e Method or procedure used for calibration, 

e Dates of last and next calibrations, 

e Person performing the calibration, 

e Values obtained during calibration, 

e Nature and traceability (if applicable) of the calibration standards, 

e Records may be electronic. 
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*When found to be out of calibration and the situation cannot be immediately addressed, then instru- 
ments are taken out of operation and tagged as such until the situation is corrected (see Section 13). (9.7) 


5.0 


MAINTENANCE 


5.1 
5.2 


*A laboratory maintenance system or program is established. (5.1) 
*Maintenance procedures are established for all measuring and testing equipment as appropriate. (9.1) 


5.3 


*Procedures are established for the creation and management of instrument maintenance records. Such 


records indicate the instrument being maintained, the dates of last and next maintenance and the person 
performing the maintenance. Records may be electronic. (7.3.2, 9.1) 


6.0 


6.1 


QUALITY CONTROL PROGRAM 


*Laboratory has established Quality Control system or practices for: (10.1) 

e Regularly testing quality control samples, 

e Timely interpretation of test results using appropriate control charting techniques, 

e Determining the in-statistical-control status of test methods in terms of the method stability over time, 
the precision and the bias. 


6.2 


*Laboratory has documented procedures for creating and maintaining records for QC samples. (7.4.1) 
QC sample records should include: 

e Sample name and source, 

e Test(s) for which it is used, 

e Assigned values (ARV) and their uncertainty where applicable, 

e Receipt date or date put into active QC use in the laboratory, 

e Expiration date of QC sample (if applicable). 


*Procedures are established for QC Sample testing frequency. (10.1.3) 


*All persons who routinely operate the test system participate in generating QC test data. (10.1.5) 


QC samples are treated as regular samples. (10.1.6) 


The laboratory uses random and/or blind testing to evaluate performance (optional). (10.1.6) 


*Procedures are established for obtaining and handling QC samples. QC samples meet the following: 
(10.2.1) 

e Stable and homogenous materials having physical or chemical properties, or both, representative of 
the actual samples being analyzed by the test method, 

e Well-characterized for the analyses of interest, 

e Available in sufficient quantities, 

e Have concentration values (or other measured characteristics) within the calibration range of the test 
method and reflects the most common values tested by the laboratory, 

e The same material is not used as both a QC sample and a calibration standard. It is not advisable to 
use the same chemical lot number for both a calibrant and a QC sample. 


6.8 


For QC testing that is strictly for monitoring the test method stability and precision, the QC sample ex- 
pected value is the control chart centerline, established using data obtained under site precision conditions. 
(10.2.1) 


6.9 


RM or CRM with independently assigned ARVs are used for QC testing intended to assess test method 
bias. These results are assessed in accordance with Practice D6299 requirements for check standard test- 
ing. For infrequent QC testing for bias assessment, refer to Practice D6617. (10.2.1) 


6.10 


*An investigation is conducted when the QC material is observed to be degrading or changing in physical 
or chemical characteristics. A replacement QC material is prepared for use. (10.2.2) 
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QC data are promptly plotted on a control chart. (10.3.1) 

e Charted data are evaluated to determine if the results obtained are without significant bias and are 
within laboratory established control limits (see Practice D6299 for guidance). 

e Charts used are appropriate for the testing conditions and statistical objectives. (May include charts 
such as individual, moving average and moving range, exponentially weighted moving average, or cumula- 
tive summation charts, as appropriate.) 

e Corrective actions are taken and documented for any datum or data that are out-of-statistical-control. 


Results for test samples are not reported until the QC data are assessed and the testing process is veri- 
fied to be in- statistical- control. (10.3.1.1) 


*Control charts contain the following information (10.3.1.1): 
e Test method, 

e Date QC analyses performed, 

e Initials/name of person performing test. 


Adequate training is provided to the analysts enabling them to generate and interpret the charts. (10.3.2) 


QC charts are displayed prominently near the analysis workstation or are readily available at computer 
terminal or PC station. (10.3.5) 


*Quality control charts are preventive action tools. Supervisory and technical personnel periodically re- 
view the QC charts and ensure relevant actions are taken to possibly prevent or respond to out-of- 
statistical-control situations. (10.3.4) 


*The laboratory established written procedures outlining the appropriate interpretation of QC charts and 
responses to out-of-statistical-control situations observed. (10.3.5) 


When an out-of-statistical-control situation has been identified, remedial action is taken before analyzing 
further samples. In all such cases, the QC sample is run to ensure that a satisfactory result can be ob- 
tained before analyzing unknown samples. (10.3.5.1) 


*Out-of-statistical-control situations are detected by one or more analyses of the QC sample. For these 
cases, the lab has procedures to retest samples analyzed during the period between the last in-statistical- 
control QC data point and the QC data point that triggered the out-of-statistical-control notice (or event) us- 
ing retained samples and equipment known to be in-statistical-control. If the new analysis shows a differ- 
ence that is statistically different from the original results, and the difference exceeds the established site 
precision of that test, the laboratory has procedures to decide on what further actions are necessary. 
(10.3.6) 

The laboratory uses the test performance index (TPI) to compare the precision of the laboratory mea- 
surements from the QC charts (that is, the site precision) with the published precision for the standard test 
method. (12.1) 


The laboratory periodically reviews their TPI values and takes appropriate action in accordance with their 
documented procedures. (12.2 to 12.4) 


*The laboratory has procedures for revision of QC chart parameters (as covered in detail in ASTM Prac- 
tice D6299). Control charts are revised when the existing limits are no longer appropriate. (10.4) (Guide- 
lines for when QC chart revisions are needed may include: additional information becomes available; the 
process has improved; a new QC material is initiated and the mean value is different than the previous QC 
material; there are major changes to the test procedure.) 


Procedures are established for retaining completed control charts. (7.4.2) (These procedures also ad- 
dress situations when a retained control chart is altered for any reason, to include recording the date of 
change or alteration and the reason.) 


7.0 


BIAS MANAGEMENT AND PROFICIENCY EVALUATIONS 


7-1 


*Laboratory participates in regularly conducted interlaboratory exchanges or crosscheck programs, where 
typical samples are tested by multiple test facilities using a specified (for example ASTM) test protocol. 
(11.2.1) Participation in proficiency testing is not considered as a substitute for inhouse quality control. 


(11.2.5) 


7.2 


Crosscheck results (that is, deviations from the consensus values) are monitored with control charts to 
ascertain if their measurement processes are non-biased. (11.2.3) (Could be mandatory based on regula- 
tion) 


7.3 


The laboratory assesses the precision of these exchange data against precision established by in-house 
QC sample testing to determine consistency and adequacy of performance. (11.2.3) 


8.0 


DATA MANAGEMENT 


8.1 


*Procedures are established for collecting and retaining records of original observations, calculations and 
derived data. (7.5.3) 


8.2 


*The records for each test contain sufficient information to permit satisfactory replication of the test and 
recalculation of the results. (7.5.3) 


8.3 


*All data are maintained according to laboratory, company, or regulatory agency requirements, or a com- 
bination thereof. (7.5.2) 


8.4 


*A certificate or report communicates test results and all other relevant information from the laboratory 
accurately and unambiguously. (This report may be an entry in a Laboratory Information Management Sys- 
tem (LIMS) or equivalent system.) (7.1.1) 
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preventive action taken. The corrective and preventative action reports are reviewed and approved by man- 
agement and then verified for effectiveness of the actions. 


8.5 The format and content of the laboratory certificate or report is specified by lab management and/or in 
customer agreements. 

Laboratory reports include the following: (7.1.2) (7.1.3) 

e Name and address of testing laboratory (for external customer reports), 

e Description and identification of the test sample, 

e Identification of the test specification, if any, 

e Test results with appropriate units of measure, 

e Identification of test method(s) used (including reference to any deviations), 

e Signature and job title of person(s) accepting technical responsibility for the test report laboratory 
(allowances made for electronic signatures or email reports), 

e Date of issue, 

e Other information as required by the customer. 

[Other information for consideration in the report includes: unique identification of the report (such as 
serial number) on each page of the report; name and address of the customer; order number; date of 
receipt of the test sample and date(s) of performance of test; description of the sampling procedure; any 
other information relevant to a specific test; disclosure of nonstandard test method or procedure utilized; 
measurements, examinations and derived results, supported by tables, graphs, sketches, and photographs 
as appropriate; any failures identified; minimum-maximum product specifications; statement of the 
measurement uncertainty; statement on the laboratory policy regarding the reproduction of test reports.] 

8.6 *Procedures are established for corrections or additions to a test report after issue. (7.1.4) 

8.7 *Laboratory uses reporting requirements (for example, format, significant figures, and units) specified in 
the test method or procedure. (7.2.1) 

8.8 The rounding protocol of Practice E29 is used when data rounding is employed, unless otherwise speci- i 
fied in the method or procedure. (7.2.2) 

8.9 The record retention system is documented and suits the laboratory’s particular circumstances and com- _ 
plies with existing regulations and customer specifications. (7.5.1) 

8.10 *The records are held in a safe and secure storage. A system allows locating the required documents in a) 
a reasonable period of time. (7.5.4) 

8.11 *A document control process is established. As a minimum, for laboratory procedures (for example, 
standing operating procedures, SOPs) and laboratory test procedures (if used), the document control pro- 
cess addresses version/revision identification and management approval. 

9.0 TRAINING 

9.1 *Laboratory training program addresses the following areas: safety, test methods, company policies and 
QA/QC procedures. (15.1 to 15.2) 

9.2 *Training records are maintained. (15.3) 

So n 

10.1 *Laboratory has a system to periodically review its own practices to confirm continued conformance to 
the laboratory’s documented quality management system. (11.1.1) 

10.2 *Test Method Assessments are conducted to confirm adherence to the documented test methods. The 
performance of the entire test is observed and checked against the official specified test method. Test 
methods are audited annually or as specified by laboratory management. (11.1.2) 

10.3 *Audit results are promptly documented. The assessment team reports the audit results to management 
having the authority and responsibility to take corrective action and to its management. (11.1.3) 

10.4 *The findings and recommendations of these internal audits are reviewed by the laboratory management 
and acted upon to correct the deficiencies or nonconformances. (11.1.4) 

10.5 *The effectiveness of any corrective actions taken in response to an audit is verified. The follow-up re- 
sults are documented as required by the quality management system procedures or laboratory policy, or 
both. (11.1.5) 

11.0 CORRECTIVE AND PREVENTIVE ACTIONS 

11.1 *Corrective and preventive action systems and procedures are established. (13.1) 

11.2 The situations for implementing corrective and preventive actions are defined by the laboratory and may 
include one or more of the following unacceptable situations: 

e Equipment out of calibration, 

e QC sample out-of-statistical-control, 

e Results out of specification (generally investigated by customer and not necessarily by the laboratory), 

e Outlier or unacceptable trend in an interlaboratory cross-check program, 

e Nonconformance identified in an external or internal audit, 

e Nonconformance identified during review of laboratory data or records, 

e Customer complaint. 

11.3 The corrective action procedures include an investigation to determine the root cause. (13.2) Procedures 
are available to guide in the investigation of root causes. (13.3) 

11.4 Corrective action procedures also consider the following: (13.2) 

e Determining when the test of equipment was last known to be in-statistical-control, 

e Identifying results that may have been adversely affected, 

e How to handle affected results already reported to a customer, 

e What to do if the root cause cannot be determined, 

e What to do if it is determined that the original data is correct. 

11.5 Corrective action reports are documented promptly and include the situation, root cause, and corrective/ 
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12.0 CUSTOMER INTERACTIONS 

12.1 *A procedure is established to follow-up on customer complaints or nonconformances brought to the 
laboratory’s attention by the customer. (12.1) 

12.2 *Results of any investigation and associated corrective actions involving customer complaints or inquiries 


are documented and communicated to the customer as soon as practical. (12.1) 


13.0 CONTINUOUS IMPROVEMENT 


13.1 *The laboratory has established continuous improvement goals. (5.1) 
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TABLE X2.2 Supplemental (Optional) Auditor Actions/Questions to Support Table X2.1 Checklist 


Note 1—This table is intended as a general guide to provide additional information to assist the auditor in completing the Table X2.1 Checklist. This 
table is not intended to be all inclusive nor to have every item apply to all petroleum, petroleum product, liquid fuel, and lubricant laboratories. Auditors 
are expected to use their professional judgement in applying this supplemental guide. 


1.0 Quality Management System 
1.1 e Verify that the laboratory has a documented Quality System (for example, Quality Assurance Manual—QAM) and that it complies with D6792. 
e Obtain a list of referenced and associated laboratory test procedures (LTP), laboratory procedures (LP) and standard operating procedures (SOP) and 
verify that they exist. 
e Verify that there is a management approved policy, directive, or endorsement for the goals and operations of the laboratory. 
e Obtain an organization chart for the lab and verify that there are clear lines of authority. 
e Verify that job descriptions are available for all laboratory staff positions. 
e Is there a procedure for reviewing and/or updating procedures (that is, QAM, LTPs, and LPs)? Verify that the procedure is followed. 
e Verify that a document control process exists to designate or identify the latest (up to date), official copy/version of key documents like the QAM, 
LTPs, LPs, etc. and that these are readily available to laboratory staff. (See Checklist §8.11) 


1.2 e Who is designated to oversee quality in the Lab and where is this documented? 
e Are these responsibilities covered in their job description? 
1.3 e Review the lab’s mission statement. 
e Verify that the objectives listed in Checklist §1.3 are addressed. 
1.4 e Verify that the QA Manual addresses each of these topics: sample management, data and record management, audits and proficiency testing, correc- 


tive and preventive actions, ensuring that procured services and materials meet the contracted requirements, ensuring that personnel are adequately 
trained to obtain quality results. 
e Verify that training records cover the QAM and related SOPs. 
16 e Verify that the requirements for a management review are documented in the QA Manual. 
e Verify that a recent review was performed and documented. 
e Was the review approved by management? 
e Verify that corrective actions were identified and completed. 
2.0 Test Methods 
2.1 e Verify that the laboratory has procedures for documenting, approval, and reviewing/updating test methods. 
e Verify that these procedures specifically define a process to ensure that other ASTM test methods allowed by any future amendments to regulations 
or specifications will be incorporated into the respective laboratory test procedures. 
e Verify that the version or revision of the test method (or LTP) used by the laboratory is current, meaning that it corresponds to the latest ASTM 
version, the ASTM version required by regulation or by specification. 
e Determine how the LTPs are made available to the lab staff and confirm that this constitutes “readily available.” 
e Verify from training records that the technicians have been trained on the latest test method version available in the laboratory (see Training). 
e Verify from records or other evidence that the technicians are using the respective LTPs or copy of the ASTM test method. 
2.2 e As determined above, verify that the test methods used are those stated in the respective product specifications, regulations, or customer agreement. 
e Are all tests specified in the product specifications being performed (by the product quality laboratory). 
e Verify for each test method that any LTP conforms (exactly) to the corresponding ASTM test method. 
2.3 e Determine if the laboratory is using a modified version of the specified test method. If so, has it been shown that the modified method gives results 
equivalent to those obtained by the specified method? 
e Verify that the lab has a procedure for approval, documentation and reporting of deviations from the published test method. 
e For any LTP that acknowledges deviations from the ASTM test method, verify that the approval process was followed and that the potential impact of 
the deviation was addressed. 
2.4 e Verify that the laboratory has procedures for obtaining approvals for use of alternative or substitute test method from the customer. 
e If an alternative/substitute procedure is used, verify that Practice D6708 was used to assess the expected agreement between the alternative proce- 
dure and the required method. (Note D6708—Use of this practice requires results from an interlaboratory study involving both standard test methods.) 
Sampling and Sample Management 
e Verify that sample management process is documented in QA Manual or associated SOPs or LPs. 
e Verify that this process addresses topics from 3.2 to 3.4 below. 
e Verify that this procedure has been updated as per QA system requirements. 
e Verify that the appropriate sampling and sample handling practices specific for the sample type (for example, gasoline, diesel, aviation turbine fuel, 
LPG, coke, pitch, lube oil, used oils, etc.). 
NOTE—It is the responsibility of the laboratory (and auditor) to determine the appropriate sampling and sample handling practices that apply. 
e Verify that criteria for sample acceptance are established and documented. 
e Verify that these criteria consider sample labeling, time and date sample collected, person who collected sample, sample material identification, 
sample location, sample temperature and sample containers. 
e Verify that the sample management procedures outline steps to be taken if criteria requirements are not met. 
e Verify that sample acceptance criteria are posted at sample receiving and/or that the receiving technicians are aware of the acceptance criteria. 
e Review sample receiving records to verify conformance with sample acceptance criteria. 
e Observe sample receiving to verify sample acceptance criteria are being followed. 
e Review sample tags to verify completeness of information in accordance with the lab procedure. 
e Verify that the sample management process addresses the sample login process and the process to assign unique sample IDs to samples entering 
the lab. 
e Verify that each batch of conventional gasoline produced at the refinery is assigned a batch number. Verify that each batch will have a unique sample 
identifier. 
e Verify that there are procedures for handling and identifying (with a unique ID) non-routine (unscheduled) samples 
e Verify that the sample management procedures describe how sample residuals (sample remaining in original sample bottle following completion of 
testing), extracts, digestates, duplicates, and excess sample containers are addressed within the system. 
e Verify that all samples are properly identified. 


3.0 
3.1 


3.2 
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3.3 


e Verify that the sample management procedures address items listed below as appropriate. 

e Samples are logged into system prior to analysis 

e Sample containers comply with requirements of respective test methods. 

e Facilities and equipment are available for proper storage of samples prior to analysis. 

e Samples are handled and stored in accordance with regulations and appropriate ASTM practices (for example, D4057, D5842, and D5854). 
e Are appropriate safety procedures in place? 

e Containers for volatile materials are filled only to the amount specified for the test method (for example, D86, D5191, D4057, D5842) 
e The volume of sample collected for retain conforms to specifications, SOPs or regulations. 

e Verify that the inventory or retain samples held for each product conforms to SOPs and/or regulations. 

e Retain samples are adequately secured, including off hours. Verify that retain samples are held for the appropriate period. 

e Verify that retained samples are being disposed following their maximum retention time. 


3.4 


e Verify that there are procedures to address sample disposal. (These procedures may be addressed in a site chemical hygiene plan and/or waste dis- 
posal plan.) 

e Have applicable regulatory requirements been identified? 

e Do these plans cover applicable procedures within the laboratory environment? 

e Verify that the procedures applicable to laboratory operations are being followed. 


4.0 


Calibration 


a4 


e Verify that the calibration system/program is documented (in QAM or LP). 

e Verify that the documented calibration program provides sufficient guidance to address items in this section. 
e Verify that a calibration schedule is established. 

e Verify that required calibrations are completed when needed or scheduled. 


4.2 


e Calibration procedures are established for balances and thermometers and other equipment not covered by a test method. 

e Confirm if balances are used for analytical purposes, they are calibrated using relevant ASTM procedures. 

e Confirm if thermometric measuring devices are used, they are calibrated using relevant ASTM procedures. 

e Verify that calibration procedures identified within test methods are properly reflected in the corresponding LTP (evaluated here or in 2.2). 

e Verify that analytical calibration standards are prepared appropriately. For example, there are standards for: calibration of ICP-AES and XRF instru- 
ments for metals analysis, preparation of liquid blends for use as analytical standards, preparation of low-pressure gas blends, handling of crude oil 
samples for mercury analysis, validation of process stream analyzers, calibration of moisture analyzers. 

NOTE—It is the responsibility of the laboratory (and auditor) to determine the appropriate practices for preparation of calibration standards that apply to 
their test methods. 


4.3 


e Verify that the calibration procedure addresses verification of vendor calibrated equipment/instruments. 
e Verify that such verifications are documented. 
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e Verify that calibration standards for balances and thermometers are appropriate and traceable to national standards. 

e Verify that calibration standards identified in the test methods are the ones being used and meet the specification in the test method. [This could be 
accomplished as part of the comparing LTP versus the ASTM test method in 2.2] 

e Verify that primary calibration standards or CRMs are used when specified or allowed by the test method. 

e Verify that where appropriate, values for reference materials are produced following appropriate (NIST or other standards issuing body) certification 
protocol. 

e Verify that the material analyzed in proficiency testing programs meeting the requirements of Practice D6300 or ISO 4295 are used as reference 
materials, provided no obvious bias or unusual frequency distribution of results are observed. 
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e Verify there are procedures for the storage of calibration and reference materials in a manner to ensure their safety, integrity, and protection from con- 
tamination. 

e These materials include purchased standards, chemicals, product standards, prepared standards, and working standards. 

e Verify that calibration or reference materials are labeled and stored as per documented procedures. 
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e Verify that calibration records are maintained for general laboratory equipment like balances, thermometers, etc. 
e Verify that calibrations identified for test methods are being performed at the intervals required and that the corresponding calibration records are be- 
ing properly maintained. 


4.7 


e Review the calibration records for each test method and determine whether they comply with the key components listed in D6792, subsection 7.3.1. 
e Such records usually indicate the instrument calibrated, method or procedure used, dates of last/next calibrations, person performing calibration, val- 
ues obtained, permissible tolerances, and traceability of the standards. Applies to thermometers, balances. 

.© Verify that the calibration records for calibrations specified in test methods (see 4.6 above) comply with these requirements. 
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e Determine if any instrument or equipment is out-of-calibration or are past due calibrations (for example, review calibration schedule; observations dur- 
ing walkabouts in the laboratory; review of calibration records). 
e Determine if out-of-calibration instruments are properly tagged as such. 


5.0 


Maintenance 


5.1 


e Verify that a maintenance program is documented (in QAM, LP, or SOP). 
e Verify that a maintenance schedule been established for all equipment in the lab as appropriate. 
e Determine if an equipment inventory list available and is up-to-date. 
e Determine if major instruments are included in service contracts. 
e Verify that the service performed as contracted. 
e If not under service contract, determine if arrangements have been made to get the instrument speedily repaired or use of a back-up lab for 
analysis. 
e Verify that there is a procedure or backup system in case of instrument failure. 
e Determine if there is plan for upgrading and replacing older instrumentation. 


5.2 


Verify that procedures are available for performing the maintenance identified for lab instruments and equipment. 


5.3 


e Verify that procedures are established for creation and management if instrument maintenance records. 

e Verify that maintenance records are maintained (log books used or some other mechanism). 

e Verify that maintenance records are current. 

e Verify that the maintenance records indicate the instrument being maintained, the dates of last and next maintenance and the person performing the 
maintenance. 

e Verify that logs are kept of all downtime and service problems for instruments. 


6.0 


Quality Control Program 


6.1 


e Verify that a QC program is documented (in QAM and LPs). 
e Verify that the QC program includes regular QC sample testing, timely interpretations, use of appropriate control charting techniques, and determina- 
tion of in-statistical-control status. See Practice D6299 A1.5 for control charts. 
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6.2 


e Verify that the QC Program describes the documentation and recordkeeping practices for the QC samples. 

e Verify that individual QC sample records include (as appropriate): sample name and source, test(s) for which it is used, assigned values (ARV) and 
their uncertainty, receipt date or date put into active QC use in laboratory, and expiration date of QC sample. 

e Determine who is assigned to maintain the QC sample records. 

e Verify training records. 


6.3 


e Verify that there are procedures to establish the QC Sample testing frequency for each test method within the QC program. 

e For each QC Sample program, verify that the QC Sample testing frequency has been established in accordance with the following as appropriate: 
e When site precision (see Practice D6299, section 9.1) is not established then the recommended frequency one QC out of every ten samples 
analyzed, 

e Alternatively, one QC sample is analyzed each day that samples are analyzed, whichever is more frequent, 

e When site precision is established the minimal QC frequency may be adjusted based on the TPI and the PR (see D6792Table 1). 

e For tests performed infrequently at least one QC sample be analyzed each time samples are analyzed, 

e Use the flow chart in D6792Fig. 1to determine if a higher QC frequency should be used. 

e Recalculate and reviewed TPI at least annually and adjust QC frequency as appropriate. 

e Assess whether the QC sample testing frequency provides adequate control for each test method. 

e Verify that the QC testing frequencies are followed as evidenced from the control charts. 


6.4 


Verify that all technicians routinely running a test also participate in generating QC chart data. 


6.5 


Review QC chart records to verify that there is no evidence to support that running any QC sample is given special treatment that would invalidate the 
control. 


6.6 


Verify the laboratory QC Program addresses random or blind testing schemes. A blind/random program either exists (documented) or such a program is 
not run but is explained in the QA Manual. 


6.7 


6.8 


6.9 


e Verify that procedures exist describing how QC samples are obtained and handled. 

e Verify for each QC sample that the material meets the following requirements: stable and homogeneous, representative of the actual samples being 
analyzed by the test method, source of material known and batch data available, sufficient quantity, concentration values within calibration range and 
within regulatory limits, concentration reflects the most common values tested, and not used as both a QC sample and a calibration standard. 

Verify that the expected value for the QC sample is determined in accordance with Practice D6299. Applies to all QC charts (See Practice D6299 An- 
nex A1.5). 

e Verify that there is a procedure for using RM/CRM material in QC testing to assess test method bias. 

e Verify that the results of such assessments are in accordance with D6299, Section 9.2 

e Determine if D6617 is used for infrequent QC testing for bias assessments. D6617 covers a methodology for establishing an acceptable tolerance 
zone. 

e Determine if the same sample is used for calibration and QC Sample—verify rationale. 

e Verify that an investigation is conducted when the QC material is observed to be degrading or changing in physical or chemical characteristics (lead- 
ing to the eventual replacement of the QC material). 

e Verify how QC samples are replaced. 

e Review records associated with control chart investigations? 


e Verify that the QC data are promptly plotted; 

e Compare date/time stamp for QC data with corresponding date/time stamp for entry of data on chart. 

e If captured directly into LIMS verify from the program description that the charts are available in a timely manner for evaluation. 
e Verify for each QC sample that the QC chart type used is appropriate and complies with Practice D6299 (see Annex A1.5). 

e Verify that corrective actions were taken and documented for any out-of-statistical-control datum. 


e Verify that the QC program procedure assures that QC results are assessed prior to reporting results on production samples. 
e Validate that this process is followed. 


Verify that the control charts (see LIMS chart) list the test method, date QC analyses performed, and initials/name of person performing test. 


Verify from training records that analysts are trained to generate and interpret control charts. 


Verify that the control charts are readily available. [Readily available can be defined as the time for the data to be plotted on the control chart and 
showing any flags within the time frame necessary to decide whether the test method is in-statistical-control and can be used to analyze product.] 


e Verify that a review of control charts by supervisory personnel is defined in the QC program. 
e Verify that these reviews are carried out and documented? 
e Verification of timely reviews might be determined from chart trends or out-of-statistical-control events that are not addressed in timely manner. 


6.17 


e Verify that procedures are available for outlining the appropriate interpretation of QC charts and response to outof-statistical-control situations. 
e Verify that these procedures generally comply with D6299 (see Annex A1.5) or otherwise provides adequate control for test method. 
e Verify that the procedures are followed and documented? 


e Verify that corrective actions are taken for any out-of-statistical-control event prior to analyzing samples. (See 6.12) 
e For out-of-statistical-control events, verify that a QC sample is run following corrective actions. Determined from inspections of QC chart data for audit 
cycle 


e Verify that the lab has a procedure to address the decision whether or not to retest samples analyzed during the period between the last in-statistical- 
control QC data and the QC data that triggered the out-of-statistical-control flag. 

e Verify that if an analysis of a retain sample shows a difference that is statistically different from the original result, and the difference exceeds the es- 
tablished site precision of that test, the laboratory has procedures to decide on what further actions are necessary. 


6.20 


e Determine if the laboratory uses the TPI to compare the precision observed from the QC charts (that is, the site precision) with the published preci- 
sion for the standard test method. 
e Determine if the laboratory uses the PR along with TPI to estimate the influence that non-site specific variations has on the published precision. 


e Verify that the laboratory periodically reviews their TPI values and takes appropriate action in accordance with their documented procedures. 

e How often does this data get reviewed? 

e Verify that the laboratory compares the site precisions (from QC charts) versus the test methods repeatability and reproducibility. 

e Verify that the laboratory investigates situations where QC chart precision is significantly worse than the test method reproducibility to determine the 
root cause for this performance so that corrective action can be undertaken if necessary. 


e Verify that the laboratory has procedures for revision of QC chart parameters (as covered in D6299, Section 8.6). 

e Confirm that these procedures include guidelines for when QC chart revisions are needed. These may include: additional information becomes 
available, the process has improved, new QC material is initiated and the mean value is different than the previous QC material, or there are major 
changes to the test procedure. 

e Verify that QC chart parameters (limits) have been revised when data indicated the need. 


6.23 


Verify that there are procedures for retention of completed control charts and they are followed. 


7.0 


Bias Management and Proficiency Evaluations 
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TA 


e Verify that there is a process or procedure covering the participation in PT programs, the analysis and interpretation of the statistical reports, investi- 
gation of out-of-statistical-control or warning flags, and corrective actions. 

e Verify that the data analysis and interpretation process references appropriate tools described in Guide D7372 (subsection 5.2.1) 

e Verify that the laboratory participates in a PT program like the ASTM ILCP. 

e Verify that the laboratory provided PT results for all the test that they run. 

e Verify that the lab maintains QC Charts (or other appropriate QC tools) for the test methods covered in the PTPs as required by regulation. 

e Verify that participation in a PT program is not a substitute for in-house QC. 


e Verify that the lab follows their process to monitor and evaluate their PT results as presented in the PTP reports in accordance with Guide D7372. 
e Verify that the lab has taken corrective actions as appropriate. 


Verify that the laboratory assesses the precision of these exchange data against precision established by in-house QC sample testing to determine con- 
sistency and adequacy of performance (see Practice D6299, subsection 7.6.2) 


Data Management 


Verify that there are procedures (in QAM or LP) dealing with the collection and retention of laboratory generated data. Verify that there are procedures 
for maintaining records containing the results of tests performed to meet fuels regulations - see 40 CFR 80.101(i). 


Verify from a review of the records maintained for each of the regulatory test methods that sufficient information are recorded and maintained to permit 
satisfactory replication of the test and recalculation of the results. 


e Verify that the lab is maintaining all the data records required by regulations and company requirements. 
e Verify that RVP results are reported to the nearest hundredth psi unit (for example, 0.01 psi). 
e Determine if all required records kept for required period of time? 


Verify that the final/released reports are clear, unambiguous, and accurate. 


e Verify that there are laboratory procedures for accurately reporting test values and properties as required by SOPs and regulations. 
e Determine if there are any key items missing from these reports. 


Does the procedure for correcting an issued report provide sufficient guidance? 


Do the test units, significant figures and rounding conventions meet regulatory requirements and/or test method? 


See 8.7. 


e Verify that there is a process (QAM or relevant LPs) for addressing retention of records and that the process is compliant with company policy and 
regulatory (40 CFR 80) requirements. 
e Verify that the record retention process (in 8.9.1) ensures that original records concerning tests of gasoline properties are maintained for five years 
from the date the test was performed. 


e Verify that the record storage facility provides safe and secure storage. 
e Verify that records are held for the minimum period and that they can be retrieved in a reasonable time. 


e Verify that a document control process is in place. Verify that SOPs and LTPs have version/revision numbers or are dated and that they are approved 
by laboratory management. 


Training 


e Verify that the laboratory has procedures to address staff training and covers key areas such as safety, test methods, company policies, and QA/QC 
procedures. 

e Verify that the training program identifies any external certifications required for certain staff functions. Verify that these certifications are current and 
available for each case. 

e Verify that the training program addresses training for all staff performing testing or interpreting data. A matrix showing which technicians perform the 
various test methods would be useful in ensuring that the appropriate test method procedure training was conducted for each staff member. 

e Verify that training being conducted as scheduled or as needed. 

e Determine if any of the training requires a verification test or other qualifier. 

e Determine if external courses used to supplement internal training. Verify that course description and training certificates are available. 


9.2 


Verify that training records are maintained and are complete. 


10.0 


Assessments 


10.1 


e Verify that there is a process for internal assessment of laboratory operations and that it addresses the following: identification, qualifications and/or 
training of lead assessors and assessors, establishing an assessment schedule or frequency, assessments of contract laboratories used by refinery, 
assessment reports, management review of reports, verification of effectiveness of corrective actions. 

e Verify that the process is followed. 


10.2 


e Verify that there is a process for conducting Test Method Assessments (TMA) and that it addresses the following: observing the performance of the 
entire test and checked against the corresponding LTP and/or ASTM test method, TMA schedule for all regulatory tests, TMA assessor qualifications 
and training. 

e Verify from records that TMAs are conducted as scheduled. 


10.3 


e Verify from a review laboratory and test method assessment reports that they are complete and were timely. 
e Verify that corrective actions were identified for any deficiencies. 


10.4 


Verify that the laboratory assessment and TMA reports have been reviewed by management and that any corrective actions were completed. 


10.5 


Verify that the effectiveness of corrective actions from assessments and TMAs are verified and documented by the laboratory. 


11.0 


Corrective and Preventive Actions 


11.1 


e Verify that there are procedures to provide adequate guidance for addressing corrective and preventative actions. 
e Verify that these guidelines include an investigation to determine root causes. 
e How are CARs tracked? 


11.2 


e Verify the situations for implementing corrective and preventive actions are defined by the laboratory. 

e These may include one or more of the following unacceptable situations: equipment out of calibration, QC sample out-of-statistical-control, results out 
of specification (generally investigated by customer and not necessarily by the laboratory), outlier or unacceptable trend in an interlaboratory cross- 
check program, nonconformance identified in an external or internal audit or TMA, nonconformance identified during review of laboratory data or 
records, and customer complaint or inquiry. 

e Verify that target dates established for completing corrective actions. 


Verify that corrective action reports include investigations of root cause. 


e Determine that the lab’s corrective action procedures (for QC charts) consider the following: determining when the test of equipment was last known 
to be in-statistical-control, identifying results that may have been adversely affected, how to handle affected results already reported to a customer, 
what to do if the root cause cannot be determined, what to do if it is determined that the original data is correct. 

e Verify that completed CARs include investigations on these items. 


e Verify that corrective action reports are documented promptly and include the situation, root cause and corrective/preventive action taken. 
e Verify that management reviews and approves CARS. 
e Determine that corrective actions are verified for effectiveness of the actions. 


Customer Interactions 
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TABLE X2.2 Continued 


12.1 
e Customer complaints may be handled as a category of CARs. 


e Verify that a procedure is established to follow-up on customer complaints or nonconformances brought to the laboratory’s attention by the customer. 


12.2 e Verify documentation of customer inquiries. 
e Was outcome of any investigation or corrective action reported back to the customer? 
13.0 Continuous Improvement 
13.1 Verify that the laboratory has established continuous improvement goals. 
13.2 Determine how often these goals are reviewed. 


TABLE X2.3 Example Weighted Scoring Practice 


Checklist Section Title 


Max Points“ Weighted, % 


1 Quality Assurance System 
2 Test Methods 

3 Sampling and Sample Management 

4 Calibration 

5 Maintenance 

6 Quality Control Program 

T Bias Management and Proficiency Evaluations 
8 Data Management 

9 Training 

Assessments 

Corrective and Preventive Actions 

Customer Interactions 


13 Continuous Improvement 


10 


^A Maximum points depend on the number of items in the checklist section and the scoring profile selected. 


X3. COMPARISON OF REPEATABILITY, REPRODUCIBILITY, PRECISION RATIO AND VARIANCE RATIO TEST OF VARI- 
OUS ROUND ROBINS 


X3.1 Practice D6300, subsection A1.7 on Variance Ratio 
Test (F-Test), provides a detailed discussion of how to deter- 
mine when significant bias exists for two data sets using the 
variance ratio. Both the variance ratio, F value and precision 
ratio, PR were calculated for 38 round robin data sets. 
Generally, the correlation between F and R/r is not statistically 
significant to suggest that PR could be used to accurately 
predict the existence of laboratory-laboratory bias for a given 
test method. However, this practice is not intended as a detailed 
statistical analysis of bias between laboratories, rather, the 
purpose of this practice is to provide some general guidelines 
for assessing the performance on a laboratory. 


X3.2 Generally, for a typical ASTM test method (for 
example, a typical number of laboratories, six or more, and a 
typical number of samples studied, ten or more) a F value of 5 
or greater exceeds the 5 % critical value given in Practice 
D6300, Table A1.6 on critical 5 % values of F, suggesting a 
bias exists between the laboratories. In addition, when the PR 
value is equal to or greater than 4, the F value is greater than 
5. This suggests that some laboratory bias may exist in the test 
method’s reproducibility statement. This observation was the 
rationale for selecting equal to or greater than 4 as the criterion 
for switching to more severe performance assessment criteria. 
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X3.3 The relationship of repeatability, Reproducibility and 
Site Precision as it relates to performance assessment criteria of 
a test method with PR<4 for a laboratory is represented in Fig. 
X3.1. This figure illustrates that a laboratory may have a site 
precision less than Reproducibility and is similar in magnitude 
to the published method’s repeatability. 


X3.4 In Fig. X3.2, there is a similar relationship of 
repeatability, Reproducibility and Site Precision for a test 
method with PR>4 as shown in Fig. X3.1. However, the 
illustration shown in Fig. X3.2 has performance assessment 
criteria for when PR<4 and PR>4 applied to demonstrate the 
difference between these two criteria. 


X3.4.1 Reviewing Fig. X3.2, a laboratory may have a site 
precision similar to the test method’s reproducibility, that is 
significantly greater than the published methods repeatability, 
but based on the PR<4 performance assessment criteria, is still 
considered to be generating acceptable results. Using the PR>4 
performance criterion forces acceptable site precision to be 
more evenly distributed between repeatability and reproduc- 
ibility so that a more thorough review of the lab performance 
may be assessed. 
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TPI<0.8 


FIG. X3.1 Comparison of Reproducibility, Repeatability and TPI Guideliens for Action for a Test Method with PR<4 


Using PR<4 |i Using PR > 4 
Guidelines for Action} į | Guidelines for Action 


TPI<0.8 i TPI<1.6 


FIG. X3.2 Comparison of Reproducibility, Repeatability and TPI PR<4 and PR=4 Guidelines for Action for a Test Method with PR=4 
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SUMMARY OF CHANGES 


Subcommittee D02.94 has identified the location of selected changes to this standard since the last issue 
(D6792 — 13) that may impact the use of this standard. (Approved May 1, 2017.) 


(1) Changes made throughout to better align this standard with 
industry best practices, and to include the contents of Guide 
D7776. 


ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned 
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk 
of infringement of such rights, are entirely their own responsibility. 


This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and 
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards 
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the 
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should 
make your views known to the ASTM Committee on Standards, at the address shown below. 


This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, 
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above 
address or at 610-832-9585 (phone), 610-832-9555 (fax), or service@astm.org (e-mail); or through the ASTM website 
(www.astm.org). Permission rights to photocopy the standard may also be secured from the Copyright Clearance Center, 222 
Rosewood Drive, Danvers, MA 01923, Tel: (978) 646-2600; http:/www.copyright.com/ 
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